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This document is the first comprehensive statement on 
research in education issued by the National Educational Research Policy and 
Priorities Board (NERPPB) that draws on systematic inquiry over the past 4 
years. Recommendations are made for legislative and administrative changes to 
improve research, development, and communication of research activities 
sponsored by the federal government. Two important trends provide the context 
for these recommendations. The first is the increasing number of students "at 
risk" in the school districts that are least able to marshal the resources to 
help them. The second is the accelerating growth of information technology. 
Two conclusions that emerge from educational research are that research has a 
proven record in educational improvement and that federal support for 
education is an investment that must be expanded. Twenty-three specific 
recommendations are made to work toward the following goals: (1) priorities 

are set and activities are problem- centered; (2) high standards of quality 
are created and upheld; (3) work is collaborative and rigorous; and (4) the 
research mission is congruent with resources. (Contains 16 references.) (SLD) 
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authorized by the Educational Research, Development, Dissemination, and 
Improvement Act of 1994. The Board works collaboratively with the Assistant 
Secretary for the Office of Educational Research and Improvement (OERI) to 
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research, development, dissemination, and the activities of the Office. The 
Board regularly reviews, evaluates, and publicly comments upon the implemen- 
tation of its policies by the U.S. Department of Education and the Congress. 

The Secretary of the U.S. Department of Education appoints members of the 
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educators, and individuals who are knowledgeable about the educational needs 
of the United States. 



Appointed Members 

Kenji Hakuta, Chair 
Professor 

School of Education 
Stanford University 
Stanford, CA 

Patricia Ann Baltz 
Teacher 

Camino Grove Elementary School 
Arcadia, CA 

James E. Bottoms 
Senior Vice President 
Southern Regional Education Board 
Atlanta, GA 



»■ 5 



Investing in Learning 



ERJC 



iii 



National Educational Raaa ar c h Policy and Priortttw Board 



Jomills H. Braddock, II 
Professor and Chair 
Department of Sociology 
University of Miami 
Miami, FL 

John T. Bruer 
President 

James S. McDonnell Foundation 
St. Louis, MO 

Ann B. Clark 
Principal 

Vance High School at Governors’ Village 
Charlotte, NC 

Edmund W. Gordon 
Professor Emeritus 
Yale University 
New Haven, CT 

Paul D. Goren 

Director, Child and Youth Development 
Program on Human and Community Development 
The John D. and Catherine T. MacArthur Foundation 
Chicago, IL 

Sharon L. Kagan 
Senior Associate 

Bush Center in Child Development and Social Policy 
Yale University 
New Haven, CT 

Glenda T. Lappan 
Professor 

Department of Mathematics 
Michigan State University 
East Lansing, MI 



& 



to 



InvMtlng In Lacmlng 



National Educational Research Policy and Priorities Board 



Robert W. Marley 
Adjunct Professor 

Department of Health and Physical Education 
Wichita State University 
Wichita, KS 

Joyce Muhlestein 
Project Director 

Utah Parent Information Resource Center 
Salt Lake City, UT 

Alba A. Ortiz 
Professor 

College of Education 
University of Texas at Austin 
Austin, TX 

Claire L. Pelton 

Director of Academic Services, Western Office 
The College Board 
San Jose, CA 

E. Lea Schelke 
Teacher 

Trenton High School 
Trenton, MI 

Ex Officio Members 

Assistant Secretary, Office of Educational Research and Improvement, 

U.S. Department of Education 
Director of Research, U.S. Department of Defense 
Director of Research, U.S. Department of Labor 
Director, National Science Foundation 
Director, National Institutes of Health 
Chair, National Endowment for the Arts 
Chair, National Endowment for the Humanities 
Librarian of Congress 

Director, Office of Indian Education Programs, U.S. Department of Interior 



z 



Investing in Learning 



ERJC 



Foreword 



The National Educational Research Policy and Priorities Board (NERPPBJ is 
charged by Congress to advise the United States on the federal educational 
research and development effort. Since its establishment by the Educational 
Research, Development, and Improvement Act of 1994, the Board has under- 
taken systematic investigation on the dimensions and scope of educational 
research and development, much of which has been carried out under the 
auspices of the Office of Educational Research and Improvement (OERI) of the 
U.S. Department of Education and by its predecessor, the National Institute of 
Education. We have emerged from this study more firmly convinced than ever 
of the importance of sound, cumulative research in education. 

At the same time, we must acknowledge that there are serious shortcomings in 
the present research and development system in education— in its funding, 
structure, organization, approaches, and even its goals and objectives. Apart 
from our own firsthand observations of OERI and other federal agencies, we 
have consulted widely with the key stakeholders— educators, families, and 
researchers— about the changes and improvements that will correct our course 
and lead us to the advancements that we seek. 

This document is the first comprehensive statement on research in education 
that draws on our own systematic inquiry over the past 4 years. We have made 
recommendations for legislative and administrative changes that we are con- 
vinced are necessary to improve the research, development, and communication 
of research activities sponsored by the federal government. We hope these 
findings will be helpful and constructive, particularly in light of the consider- 
ation by this Congress of the authorization of the Office of Educational Re- 
search and Improvement. 

Readers are encouraged to respond and to address their comments to 
National Educational Research Policy and Priorities Board 
80 F Street NW, Suite 100 
Washington, DC 20208-7564 
E-mail: nerppb@ed.gov 

Kenji Hakuta 
Chair 

National Educational Research Policy and Priorities Board 
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Summary of Recommendations 

Goal : Priorities Are Set and Activities Are Problem- 
Centered 

1. Student achievement— The priority for research in education must be a 
high level of achievement for all students, and, within that domain, the 
initial emphasis should be on reading and mathematics achievement. 

2. Reading, second language learning, and mathematics— Recent reports from 
the National Research Council, Preventing Reading Difficulties in Young 
Children , and Improving Schooling for Language-Minority Children: A 
Research Agenda, synthesize strong bodies of research knowledge. A 
similar study on mathematics is currently under way. In each case, 
research is needed now to analyze how the results of our knowledge can 
be implemented in school programs and what factors lead to success or 
to difficulties. In reading, research is needed on how students become 
facile at reading complex text as they transition to advanced academic 
subjects such as history, social science, mathematics, and science. Re- 
search in both short- and long-term effects of specific education interven- 
tions for English language learners is needed, as well as techniques of 
assessment to measure competence, and transition points, from the first 
oral language to English; from oral language to literacy; and from lit- 
eracy to the academic discourse of specific disciplines. In mathematics, 
research is needed on why students have so much trouble making transi- 
tions (e.g., from concrete objects to more abstract ideas), understanding 
formal representations, multiplicative reasoning, and essential math- 
ematical and statistical concepts, such as chance, randomness, and prob- 
ability. 

3. Organization for learning out of school— To take advantage of different 
learning environments in which children from impoverished back- 
grounds often display more competence than in school settings, research 
is needed to design and test different models of after-school and summer 
programs to motivate, engage, and benefit children of low-income 
families. Work is also needed on types and features of after-school 
opportunities that most effectively motivate academic achievement and 
positive sejf-estimations; and how to design and test different models of 
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collaboration between schools and community groups dedicated to 
providing strong learning environments for poor children. 

4. Organization for learning in school — Retention, pull-out remediation, 
tracking, and segregated special education programs that stratify by race, 
class, and gender opportunities to learn do not result in high achieve- 
ment for all students. A more complete inventory of knowledge about 
effective practices for teaching academically challenging curricula with 
groups is needed, both for school populations in general, and for hetero- 
geneous groups in particular. Research is needed on questions of time for 
children to master challenging curricula, supportive school structures, 
and expectations for the breadth and depth of content. Within each of 
these is the question, do students from middle-income families, as well as 
from low-income, ethnic, and linguistic minority backgrounds, benefit 
from each organizational practice? An important area of inquiry is 
whether there are academic benefits to classroom diversity — does diver- 
sity improve subject-matter learning? 

5. Linking changes in teaching practice with improved student learning— 
Information is needed that can guide teachers and institutions who want 
to change their educational practice, particularly to reduce inequities in 
the opportunities of students who differ in socioeconomic status, ethnic 
background, and gender to learn successfully. This is especially impor- 
tant regarding the achievement of deep intellective competence advo- 
cated in current educational reforms. Such research would examine 
fundamental issues about the nature of teaching and learning, including, 
but not limited to, the importance of the skills and knowledge of teach- 
ers. We need to expand our knowledge and understanding of teaching 
practices, including teaching tools, such as assessment, that are successful 
with students who bring different cultural resources to their own and 
others’ learning. Research would examine, much more so than in the 
past, issues of what it takes to do effective and successful teaching with 
diverse populations of students. 

6. Linking teachers' professional development and teaching practices— Re- 
search is needed to understand what effective teachers do and how they 
do it. Successful teaching involves not only the exercise of skills and 
application of knowledge but also flexible improvisational adaptation in 
the classroom. Research is needed to understand the roles of increased 
knowledge and comprehension of subject-matter concepts and methods, 
the role of thorough understanding of the curriculum in the subject both 
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at the level one is teaching and in relation to other disciplines and 
grade levels, as well as the role of understanding students’ learning in 
improving teaching practice. Research should also investigate how 
the structure of teachers’ work supports or hinders their “on-the- 
job” learning and what kinds of abilities are learned in particular 
situations that can transfer to other settings. 

7 . U nderstanding and supporting successful professional development— 
There is need for a better understanding of teachers’ development as 
“professional learning,” and of teaching as a “learning profession.” 
The model of learning how to teach, which is prevalent today, 
namely, that knowledge goes in during teacher education or profes- 
sional development and then comes out in the teachers’ own class- 
rooms, does not account for the engagement of teachers themselves 
in improving the practice of their profession. What teachers need to 
learn to put reforms in place is not separable from their actual 
teaching practices or from the development trajectories of their 
careers. Research needs to examine ways in which people of diverse 
cultural, ethnic, and socioeconomic backgrounds are attracted to 
careers in teaching. How can professional development resources 
help support diversity in the teaching profession as well as improve 
practice? Further research on teachers’ communities of practice is 
needed, building on findings that norms of responsibility and colle- 
gial efforts at professional problem solving are the most critical 
factors in improvement of teaching and learning. 

Goal: High Standards of Quality Are Created 
and Upheld 

8. Standing panels — Standing panels should be established to review 
proposals for each OERI institute. These would be comprised of 25 
to 30 members, but with some overlapping membership, so that 
problems that cross boundaries can receive informed attention and 
that members of one panel with special knowledge could be invited 
to serve on another. 

9. Panel membership — Panels should represent a broad range of perspec- 
tives. They must include members with strong disciplinary and 
methodological expertise. Across OERI panels, gender, race, 
ethnicity, and geographic diversity must be respected. Panel mem- 
bers should generally be nationally recognized figures. 
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10. Standards for panelists— The Board continues to support peer review 
standards adopted by OERI, with Board approval, which stipulate that 
all reviewers meet three criteria: demonstrated expertise, including 
training and experience in the subject area of the competition; in-depth 
knowledge of policy or practice in education; and in-depth knowledge of 
theoretical perspectives or methodological approaches in the subject area 
of the competition. 

1 1. Distinguishing between field-initiated and directed competitions— OEBA 
should distinguish between field-initiated competitions and those that 
are directed, rather than trying to combine the two. 

12. Funding for peer review— The allowable percent of funds to support peer 
review should be increased so that the necessary standing panels and 
logistical support can be provided. 

13. Definitions— The term “research” should be more narrowly defined than 
it is in the 1994 law so that the boundaries of focused competitions for 
research can be limited. Research should emphasize basic research in 
education as well as investigations, experiments, and inquiry to develop 
new knowledge or apply tested knowledge. It should exclude develop- 
ment, planning, and demonstrations. The term “national significance” 
needs to be clarified through regulations or in legislative language so that 
reviewers understand that it includes research opportunities, not only 
important problems identified by educators. 

Goal : Work Is Collaborative and Rigorous 

14. Collaboration across federal agencies— The Assistant Secretary for OERI 
should extend efforts to join with other federal agencies, and perhaps 
foundations, to collaborate on common agendas. 

15. Coordination of research within the U.S. Department of Education— The 
Secretary should encourage, and the Assistant Secretary for OERI 
should provide special attention to performing a visible and constructive 
role in collaboration and coordination of education research within the 
Department. 

16. Linking ongoing research and practice-related efforts— The Assistant Secre- 
tary should develop constructive means through which OERI, the 
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Research and Development Centers, and the Regional Educational 
Laboratories can function closely together to maximize their collec- 
tive impact. 

17. Synthesis activities — OERI should support synthesis activities across 
all-important fields of educational research, summing up progress 
continually and drawing implications for policy and practice. 

18. OERI staff— The Assistant Secretary needs to determine the responsi- 
bilities most appropriate to accomplish the research functions of 
OERI, both to advance its work and to attract, retain, and continu- 
ously nurture staff with the requisite training and opportunities. 

19. Collaborative problem-solving research and development— OERI 
should begin development of a new design for research that would 
focus explicitly on solving specific current problems of practice and 
at the same time be accountable for developing and testing general 
principles of education that can “travel” to locations beyond those in 
which the research is done The central idea is to develop a system of 
support for projects in which professional researchers and profes- 
sional educators share in the accountability for achieving success in 
improving educational practices and outcomes. These projects may 
also include program developers, curriculum specialists, or policy 
specialists. Initially, this effort might be launched through a “work- 
ing group” to assist in designing specific parts of the priority research 
agenda for which this model would be best suited, the role of OERI 
and outside contractors or grantees, coordination across multiple 
sites, expectations as to scale and length of commitment and the like. 

Goal: Mission Is Congruent with Resources 

20. Funding research in education— Funding for educational research 
must be increased dramatically. An interim target should be to reach 
the level of 0.5 percent of our nation’s expenditure for elementary 
and secondary education, about $1.5 billion annually, which was the 
amount proposed by the President’s Committee of Advisors on 
Science and Technology. It would be feasible to reach this target 
over a 5-year period. 
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21. Aligning resources and mission— Over the next 5 years, OERI should 
work to make the resources and missions better aligned. The missions 
must be matched with money. 

22. Research supervision— Some focal point should be created by Congress 
for research leadership that can span across administrations. 

23. Allocation requirements— The allocation provisions for institutes and for 
types of support in the Educational Research, Development, Dissemina- 
tion, and Improvement Act of 1994 should be removed. 
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I. Introduction 



Research in Education 

America’s students fall far short of academic achievement levels that policy 
leaders, parents, the public, and the media believe are necessary to equip them 
for living, for active citizenship, and for productive employment in the 21st 
century. This is perhaps the most frequently repeated message Americans hear, 
see, or read about education. Rarely, however, is our nation told that research 
has proved that it can make a difference in the practice of education so that 
more students will learn effectively. Yet that has been the record. Members of 
the National Educational Research Policy and Priorities Board believe that 
investments in education research are justified, and, more important, that 
commitment to substantial growth in federal support of these investments is 
urgent. Without these investments, guidance for education policies and practices 
is too often left to uninformed opinion and unreasoned prejudice. 

The Board has reached these conclusions as a result of experiences shared by the 
members since its establishment in March 1995. It has carried out the duties 
assigned in law, many of them in collaboration with OERI, commissioned 
studies and evaluations, and consulted with many individuals. Representatives 
of scholarly organizations, schools, advocacy groups, and governmental agen- 
cies have briefed the Board about how research is carried on, what findings have 
been reported and analyzed, and how those findings have been put to use. 
Established under the Educational Research, Development, Dissemination, and 
Improvement Act of 1994, the Board has had responsibility to develop policies 
and priorities for research in education. This policy paper is the Board’s first 
comprehensive statement on the federal effort in education research. 

Two important trends provide the context for the nation’s challenge to helping 
our youth reach acceptable educational performance standards and outcomes. 
The first of these is the rapid increase in the number of students “at risk” in 
school districts that are least able to marshal the human and financial resources 
to meet their needs. The demographic and geographic characteristics of pro- 
jected growth in the youth population over the next 30 years suggest that 
virtually all of it will be concentrated in these “at risk” areas. But one should 
not suppose that,p^r national challenge is confined to children at risk in dete- 
riorating urban school systems. As the recent Third International Mathematics 
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and Science Study (TIMSS) has made clear, deficiencies in mathematics and 
science skills and knowledge extend to all students at all levels of relative profi- 
ciency, including those who live in well-financed districts with mostly majority 
population. 

A second important contextual trend is the accelerating onrush of information 
technology. Already reaching into many aspects of the lives of students, the 
new technologies are increasingly shaping formal education, for better or 
worse, and reemphasizing disparities between the “haves” and the “have-nots.” 
This trend is not simply a matter of access to computer hardware and software, 
vital as these factors are; it is also about the critical need to plan and integrate 
new technology into teaching and curricula, so as to expand and extend student 
learning. 

These interacting trends represent a problem of immense national significance. 
The national educational enterprise in its many forms is widely and correctly 
understood to be a central device for the development of the knowledge, skills, 
and perspectives necessary to the success of our economy and the well being of 
our society. Education is also part of the glue that helps to bind us together as a 
community and a bridge across our many differences. Meeting these new chal- 
lenges requires more than good will, energy, and resources. It also requires 
putting what we know to work and expanding our knowledge base so that our 
capacity to meet the challenges will be expanded. Trying to implement our 
hopes and our goals without careful research, testing, and development is likely 
to increase our frustrations without improving our performance. 

In no way is research in education a quick and effortless path to success. Over 
the years, there have been many hard lessons to be learned. Educational im- 
provement occurs slowly and in small increments no matter how powerful the 
research base behind it. Deep disagreements among prominent researchers are 
continual and perhaps inevitable. Professional educators have rarely become 
enthusiastic consumers of research. Weak designs and measures, combined with 
these professional doubts and disputes, have produced too many research results 
whose values and political implications are more prominent than their scientific 
validity. The educational research system has had powerful constraints and 
limitations on it, which have hindered numberless researchers and projects. The 
wonder is that educational researchers have been able to accomplish what they 
have. 



ERIC 






Introduction 



Yet the accomplishments of research in education have been significant and 
their influence on education often wide. This perspective has been summa- 
rized in the report from a June 1998 conference on National Directions in 
Education Research Planning, which the Board sponsored jointly with 
OERI: 



Educational research has been used time and again, at critical junc- 
tures, to improve teaching and learning. Important examples range 
from John Dewey through constructivism, to Edward L. Thorndike 
through behaviorism and educational testing. . . 

Education research has supported the design and evaluation of many gov- 
ernmental programs at all levels. Studies of learning and school organiza- 
tions have had a major impact on teaching, assessment, and education 
reforms. Three recent reports from the National Research Council (NRC) 
that sum up what has already been learned and how it might be used in 
education include: Preventing Reading Difficulties in Young Children ; Im- 
proving Schooling for Language-Minority Children: A Research Agenda-, and 
How People Learn: Brain, Mind, Experience, and School. A recent, multifac- 
eted set of research reports, the Third International Mathematics and Sci- 
ence Study (TIMSS), is the latest, most comprehensive, and most significant 
of a generation’s progress in building comparative international assessments 
of learning and instruction. The widely acclaimed “Success for All” and 
projects of the New American Schools are demonstrating the practical and 
powerful effects that research can have when it is systematically applied in 
the classroom. 

Our first conclusion from these and many other examples is that research has 
a proven record in education. Our second is that federal support for education 
research is an investment that must be expanded several-fold. Others have 
come to this second conclusion as well. For example, in a 1997 report, a 
panel of the President’s Committee of Advisors on Science and Technology 
(PCAST) called for sharp increases in education research appropriations— 
reaching 0.5 percent of U.S. spending for elementary and secondary educa- 
tion. The report of the National Directions conference agreed that greater 
funding for research in education would be wise and productive: 

The prospect is that more confidence and opportunity could pay off 
handsomely, if the support is strategically provided. 
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The National Educational Research Policy and Priorities Board not only con- 
curs, it has created this policy statement to describe critical elements of a strate- 
gic design for the federal government’s role in education research. 

Role of the National Educational Research Policy 
and Priorities Board 

Congress created the Board as an external policy setting and advisory body for 
research in education. The Board sets priorities and approves standards. It also 
conducts reviews of OERI work, serves in a liaison capacity with the education 
research field and the public, and has responsibilities to strengthen the educa- 
tion research and development system. It enjoys substantial independence in 
gathering information, commissioning consultants, meeting with representa- 
tives of the education research system and consumers of that system, and, 
perhaps most importantly, communicating with the American public and 
Congress about education research. Among its specific responsibilities are the 
following: 

• provide guidance to Congress in its oversight of OERI; 

• review regularly, evaluate, and publicly comment upon, the imple- 
mentation of Board recommended priorities and policies by the 
Department and the Congress; and 

• advise the (people of the) United States on the federal educational 
research and development effort. 

The operations of the Board are required to be collaborative— carried out in 
concert with the Assistant Secretary as well as with researchers, teachers, school 
administrators, parents, students, employers, and policymakers. Indeed, the 
concept of collaboration is emphasized both in the statement of Board responsi- 
bilities in the law (such as in policy and priorities setting) and in the categories 
for Board appointments mandated in the law, namely: 

• education researchers, nominated by the National Academy of 
Sciences; 

• outstanding school-based professional educators; and 
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• individuals who are knowledgeable about educational needs— par- 
ents, chief state school officers, local education agency officials, 
principals, members of state or local boards of education or Bureau 
of Indian Affairs school boards, and individuals from business and 
industry. 

Ex officio members in addition to the Assistant Secretary for OERI include the 
directors of research for the Department of Defense and the Department of 
Labor; the directors of the National Science Foundation, the National Institutes 
of Health, the National Endowment for the Arts, and the National Endow- 
ment for the Humanities; the Librarian of Congress; and the director of the 
Office of Indian Education Programs at the Department of the Interior. 

It is now 4 years since the first meeting of the Board. During that time the 
Board has carried out its responsibility to approve standards for review of grant 
proposals, evaluation of exemplary and promising practices, and evaluation of 
OERI’s work. It has examined and made recommendations on the peer review 
system at OERI, crucial to ensure high-quality work. It has collaborated with 
the former Assistant Secretary to set initial priorities. It has reviewed solicita- 
tions for regional educational laboratories, the research and development cen- 
ters, and other major initiatives. The Board members have also stepped back to 
evaluate their functions more broadly, both to determine the effects of what 
members have done and to assess directions for education research in the future, 
especially as the time comes for reconsideration of the authorization of OERI’s 
research and development authorities. The following pages describe the findings 
and conclusions from these studies (section II) as background for Board goals 
and recommendations (section m). 
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II. The Board's Findings 

The Board has conducted its efforts with a view of education research, develop- 
ment, and dissemination as bringing opportunity and promise to learning for 
America’s youth. As the Board has surveyed the field of education research, it 
has tried to identify the major issues in the existing system — the longstanding 
problems and the increasing need for better returns from the investments in the 
system. It has grouped these in terms of resources, balance and linkage across 
the system, and processes. 

Resources 

Longstanding problems in education research start with insufficient resources. 
Limited funds have been spread thinly over a large number of topics and prob- 
lems, rather than concentrated on fewer issues. OERI’s national research insti- 
tutes, created in 1994 to re-focus education R&D on important educational 
topics and problems, are a prime example. Notwithstanding some bright spots, 
the institutes lack sufficient internal staff to mount credible programs consistent 
with their mandates for comprehensive and high-quality work and to provide 
national leadership on critical issues. This means that the “critical mass” found 
in other research institutions to be necessary for an effective, high-quality 
program is missing. The Board’s concern about critical mass extends to the 
national educational research and development centers, which in many in- 
stances have too few resources for the work and leadership expected of them. 
The regional laboratories, as well, have immense formal missions, but only 
modest resources to achieve them. Some of these institutions have addressed the 
critical mass problem by aggressive efforts to obtain resources from other 
sources, but they all still face a mismatch between ambitious missions and 
limited resources to meet those expectations. 

If the quality, utility, and resources for education research are to improve, more 
effort, focus, and resources will be needed to strengthen the supporting infra- 
structure in three respects: 

(1) The “demand” for research should be strengthened. Building effective 
demand will require a substantial effort to educate teachers about the 
value and use of education research. That effort will succeed, however, 
only if educators are participants in the planning and execution of 
research. 
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(2) More resources can be used to attract new, highly qualified scholars 
from many disciplines to educational research and its issues. Lack of 
resources and prestige now inhibit such recruiting. 

(3) The institutions that undertake the critical work will need to be selected 
on merit, nurtured with sustaining resources and the demand for qual- 
ity, and rigorously evaluated for performance. 

The National Educational Research Policy and Priorities Board is not alone in 
its conclusion that education research is shamefully underfinanced. In 1997, the 
Panel on Educational Technology of the President’s Committee of Advisors on 
Science and Technology (PCAST) made several recommendations to improve 
the quality of education in all subject areas. The report did not focus only on 
technology as a topic of study, but as a means to strengthen content and peda- 
gogy in education, enhance professional development, and increase student 
learning. The Panel called for a large-scale program of rigorous, systematic 
research on education in general and educational technology in particular. It 
recommended an investment equal to at least 0.5 percent of our nation’s expen- 
ditures for elementary and secondary education— about $1.5 billion annually— a 
five-fold increase over what the Panel identified as the current level. That figure 
was contrasted with the pharmaceutical industry’s investment in research of an 
amount equal to 23 percent of all U.S. expenditures for prescription and non- 
prescription medications. 

Balance and Linkage 

The involvement of teachers and other education professionals in knowledge- 
building and implementation activities is stimulating new thinking about the 
design and conduct of research in education. It is increasingly clear that teacher 
acceptance of and success in revised practice is strengthened by understanding 
and involvement. This realization is leading toward efforts to seek active par- 
ticipation of teachers, schools, and districts in the research and development 
planning, and conduct and evaluation process. Some refer to this as creating 
“learning organizations.” In this role, the education professional community 
becomes vested in the objectives of the innovation and reform, provides helpful 
input in fitting concept with operational reality, and contributes a continuing 
basis for accountability and mid-course correction. 

The span of activities authorized for OERI is very broad, from fundamental 
research through large-scale demonstration and effective communication of 
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knowledge and information to the practitioner community. But that very 
breadth carries consequences when the research investments that are needed to 
improve student learning are not congruent with task. The portfolio of the 
Department is aggregated in two areas: applied research and small-scale develop- 
ment; and communication activities. The Department conducts essentially no 
basic research, and is not deeply involved in large-scale development or demon- 
stration, especially about comprehensive or standards-based reform. The cumu- 
lative science base supporting the applied agenda of the OERI and departmental 
R&D activities lacks clear visibility, which adversely affects its force and cred- 
ibility. The OERI Institutes are contributing some of the important applied 
research aimed at comprehensive or standards-based reform, testing and assess- 
ment. The Department’s participation in large-scale development has been 
generally modest. The pattern of foundation funding often mirrors the federal 
focus, although a few foundations are supporting large-scale comprehensive 
reform experimentation. Generally, foundations appear to give more focus to 
curriculum and teaching topics than the federal programs. It is the Board’s 
impression that the fit and relationship between foundation and federal funding 
are more happenstance than intentional. The absence of substantial large-scale 
development activity aimed at critical problems with rigorous research and 
evaluation is noticeable. This concern is particularly strong in light of the 
continuing difficulties of scaling up small, promising developments that require 
systemic change for widespread success. Providing knowledge of effectiveness at 
a large scale is an important insulation against faddism and insufficiently tested 
ideas. 

Turning to dissemination, OERI’s work most frequently has followed tradi- 
tional approaches, no longer believed adequate, which leave to those in need of 
exemplary practice and sound knowledge the burden of finding it. This is 
particularly true for those undertaking large-scale comprehensive reforms. Even 
with the Internet and other forms of user-friendly electronic access, the passive 
systems do not fully meet the needs of those with ambitious innovation agen- 
das, and the volume of information can overwhelm the practitioners involved. 
The more intensive efforts appear to require a combination of traditional 
dissemination, technical assistance, and short-term applied research or problem 
solving. A new set of intermediaries and adjustments in existing organizations 
are emerging to meet these needs, and dissemination must be reconceptualized 
in this broader context. 



The Board has learned that fundamental research— largely in the cognitive and 
neural sciences— is conducted in other federal agencies, most notably the Na- 
tional Institute of Child Health and Human Development (NICHD), the 
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Office of Naval Research, and to a lesser degree the National Science Founda- 
tion (NSF). There is also modest foundation support for basic educational 
research. The location and level of fundamental research is of concern in several 
different ways. It is not desirable that basic research be sponsored or conducted 
in one organizational framework, but it is important that such work be linked 
to the applied research and ultimate practitioner communities that will exploit 
and make use of its findings. Sound linkage requires that staff in applied re- 
search organizations are sensitive to implications of the findings and are quali- 
fied to design the applied and related basic research needed to push sound 
findings toward utilization. Such linkage requires continuing identification of 
application problems and unmet needs from practitioner and applied research 
communities. Efforts to link across organizational performers are occurring 
more frequently, such as recent planning work concerning a new initiative 
among OERI, NICHD, and NSF. 

The research planning processes in Education are newer and less well articu- 
lated than those for Defense and Health. Department plans set forth broad goals 
and objectives, and do a particularly commendable job of relating research and 
development efforts to mission objectives. But they are unlike the Defense and 
Health counterparts in two undesirable ways. First, they are much clearer 
about the mission and application goals than about the research goals and 
priorities. Second, there is a strong sense in the defense and health cases that the 
science base is firmly rooted, and that there is a clear sense of direction and 
cumulative learning. Moreover, in those cases, the growing knowledge base is a 
powerful determinant of both future research and operational actions. There is 
no such comprehensive sense for education research and development. These 
differences are in part attributable to the huge difference in resources among the 
three agencies, which inevitably affects the style that has been adopted. But 
more than just staff and dollar resources are at issue; there is also the question of 
whether the education research and development program is an endless series of 
small applied research projects unrelated to an evolving critical set of knowl- 
edge bases or a cohesive agenda of cumulative knowledge-building. By this 
criterion, these other federal agencies are better developed. 

Processes 

The Board’s review of the current education research system included specific 
attention to important processes by which agendas are set, support is mobilized, 
resources are allocated, and progress is achieved, assessed, and made known. 

The experiences and models in other federal R&D programs provide insights 
for assessment of OERI’s work. 
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• Balance of researcher initiatives with national focus-. The first area is 
agenda setting, in which two important objectives need to be harmo- 
nized. On the one hand, long experience suggests that R&D is ‘most 
productive if researchers are given substantial latitude in initiating 
work on their own ideas or perceived problems, generally known as 
field-initiated research. On the other hand, there is a strong and 
continuing need to give the R&D agenda a sense of national focus 
and priorities aimed at the most important gaps in our knowledge 
and the most promising research approaches. OERI has adopted 
several devices to create the desired harmony, yet the national focus 
part of the balance remains weaker than it should be. Education 
lacks a process similar to that employed in NIH, for example, that 
could ensure a continuing dialog among OERI, the Institutes, Cen- 
ters and Regional Laboratories plus other important education 
research institutions and sponsors about critical research problems 
and opportunities. 

• Respect for research as well as for policy decisionmaking : A second issue 
centers on mobilization of support. Given the importance of federal 
funds in education R&D, garnering support means the involvement 
of the federal political process. Respect for the appropriate domain 
and responsibilities of the research community and political policy- 
makers is both necessary and is accomplished in other federal R&D 
programs. Where it works well, all parties are engaged in setting 
broad objectives and parameters of the work, while leaving specific 
design and execution of the projects to the sponsoring agency and 
the research and practitioner communities. 

• Meeting high standards of quality: The third issue is development of 
appropriate research methodologies and the establishment of stan- 
dards of evidence to be applied to knowledge-building activities. 
More rigor is clearly desired and needed in education research. 
Traditional rigorous methodologies such as randomized experiments 
may not routinely be feasible, affordable, or appropriate in education 
research, though control group methodology has a continuing and 
important role in the clarification of uncertainty about critical 
research issues. Members of the Board believe that further efforts are 
needed to define appropriate research designs. The choice is not 
between randomized techniques and nothing at all. There is an 
appropriate level of rigor associated with the stage and purpose of 
the research being undertaken. 
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III. The Board's Goals and 
Recommendations for Educational 
Research 

In the 1994 “Educational Research, Development, Dissemination, and Improve- 
ment Act,” Congress set forth a powerful challenge for education research and 
for the National Educational Research Policy and Priorities Board: 

The Congress declares it to be the policy of the United States to provide 
to every individual an equal opportunity to receive an education of high 
quality ... To achieve (that) goal . . . requires the continued pursuit of 
knowledge about education through research, development, improve- 
ment activities, data collection, synthesis, technical assistance, and 
information dissemination. While the direction of American education 
remains primarily the responsibility of State and local governments, the 
Federal Government has a clear responsibility to provide leadership in 
the conduct and support of scientific inquiry into the educational pro- 
cess . . . The failure of the Federal Government to adequately invest in 
educational research and development has denied the United States a 
sound foundation of knowledge on which to design school improve- 
ments . . . (The) National Educational Research Policy and Priorities 
Board should . . . work collaboratively with the Assistant Secretary to 
forge a national consensus with respect to a long-term agenda for educa- 
tional research, development, dissemination, and the activities of the 
Office. 

As a result of its studies, and its meetings with teachers, educators, researchers, 
policymakers and others, the Board has reached consensus on four goals that are 
critical to meet the challenges expressed by Congress in the 1994 legislation. 
These goals are statements about characteristics of research in education. If the 
goals were reached, there would be a sound basis for trust in the results and 
growing support for conduct of research. The Board’s goals are: 

• Priorities are set and activities are problem-centered. 

• High standards of quality are created and upheld. 
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• Work is collaborative and rigorous. 

• Mission is congruent with resources. 

The sections below provide additional information about each of these goals 
and make recommendations for action to achieve them. 

Priorities Are Set and Activities Are Problem- 
Centered 

In the Board’s work with the Assistant Secretary to establish a priorities plan, 
and its subsequent efforts to refine and target priorities, it has found that a 
problem-centered focus for research for developing research agendas works best. 
That is, identifying real problems faced by teachers in real classrooms is the 
most understandable way to design and target appropriate research. The princi- 
pal priority should be teaching and learning and, more specifically, improved 
achievement for all students. But balanced research agenda setting must also 
give weight to identification of research opportunities— where research is poised 
for advances. The targets for action should be ones for which there is reason to 
be optimistic that research has something important to say, or could have, with 
the proper investment. 

Together, the Board and OERI sponsored a 1998 conference on “ National 
Directions in Education Research Planning' that brought together leaders and 
representatives from a dozen or more research-planning efforts under way 
among federal agencies, professional and scientific organizations. Its purpose 
was to put individuals associated with those efforts into communication with 
one another and with the educators and policymakers who could use the fruits 
of education research to enhance learning and suggest appropriate priorities and 
collaborations for current and future work. 

The overriding sense of the conference was that educational research planning 
must emphasize focus and selectivity. The Board heard a consensus among 
conference participants that education research should be concentrating its 
inquiries on those areas that the public and the profession believe are impor- 
tant, as well as anticipating problems that will become important. Among the 
conferees, the appropriate topics were identified as reading and language learn- 
ing; expanded attention to mathematics; the dynamics of teacher performance 
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and effectiveness in schools and classrooms; and new emphasis on technology 
and telecommunications; international studies; and learning in family, com- 
munity, and workplace settings. 

Student Achievement 

Members of the Board believe that the focus of research attention must be 
narrower still, in order to concentrate on something both important and 
possible. 



Recommendation 1: Student Achievement— The priority for 
research in education must be high achievement for all students 
and, within that domain, the initial emphasis should be on 
reading and mathematics achievement. 

Raising student achievement is a priority for education supported alike by 
parents, business leaders, public officials, and educators. But to attain high 
achievement for all students requires success in combating the most difficult 
and challenging issues of student performance across America. These are 
issues sharpened for us once again, recently, by international comparisons 
from the TIMSS in which both the strengths and the shortcomings in 
achievement among our youth are apparent. TIMSS data indicate that our 
younger children, age 9, demonstrate mathematics and science knowledge 
and skill at levels approximating those of children in other economically 
developed countries. But as they progress through the school system, they 
fall farther behind, so that by 12th grade, American students are among the 
lowest scoring students in the study. Before the TIMSS results were released, 
we may have been able to take comfort that our most able students ranked 
with those of other nations, but that has now been disproved as well. The 
TIMSS results repeated findings from the National Assessment of Educa- 
tional Progress (NAEP) and other sources, showing that achievement of 
children from minority backgrounds and from low-income families, on 
average, continues to lag far behind that of the majority population. 

These characteristics of student achievement— that (1) it is below levels 
experts believe necessary for maintenance and preservation of American 
democracy and for full participation in a vibrant economy in the 21st cen- 
tury, and (2) that there are unacceptably wide gaps across members of our 
population— are longstanding and have resisted well-intentioned attempts 
over many years to remedy. Members of the National Educational Research 
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Policy and Priorities Board believe that the combined efforts from researchers 
across several disciplines, developers, and teachers can make a difference if 
student achievement is the priority. 

We have called on the National Academy of Education (the Academy) to 
provide assistance in forming a research agenda around high student achieve- 
ment, one that would build on what is already known, and one that would 
capture the most promising areas for further exploration. The Academy was 
asked to create a possible agenda on a series of focused research questions. The 
Academy has suggested three strands of work. The first strand is research on 
learning, especially across transitions in children’s lives; the second on teaching 
in relation to learning as a professional practice to support student learning; and 
the third, strengthening the links between educational research and the practice of 
schooling. The first two are discussed in this section on priorities and problem- 
centered activities. The third is discussed below under collaborative work 
because it is directed toward constructing more powerful methodologies for 
conduct of research in education— not only in the area of high achievement for 
all students, but other topics as well. 

The ability of the United States to make substantial progress toward the goal of 
high achievement for all students is limited by assumptions about the nature of 
research, learning, and teaching that cause policymakers and practitioners to 
neglect important complexities associated with education. It is, for example, 
usually assumed that the results of researchers’ investigations should have 
important practical implications, whether or not the researchers are trying to 
improve educational effectiveness. Regarding students’ learning, students are 
usually assumed to learn procedures and facts independently of their compre- 
hension of the concepts and principles that make them understandable, and 
students’ learning in school is assumed to occur independently of their abilities 
and personal identities outside of school. Teachers are assumed to develop skills 
and subject-matter knowledge independently of the social and cognitive chal- 
lenges they deal with in their classrooms as they interact with students. In fact, 
however, the relations among all these factors must be better understood, and 
to do so, education must be thought of as a complex professional undertaking. 

Transitions 

In the first strand of its measures for focusing research on high achievement for 
all students, the Academy has advised us that the most critical and promising 
research questions fall into two areas. One involves transitions that students 
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must accomplish in order to progress successfully through the school curricu- 
lum. The other concerns transitions that involve the social organization of 
learning in schools and its relation to the activities of students outside of school. 

The Board recommends that research on teaching and learning in school be 
focused on critical transitions that include important developments in concep- 
tual understanding as students move through the school curriculum, and as they 
move between the school and other communities. To achieve at high levels, 
students must succeed in critical transitions that require mastery in general 
aspects of knowing and understanding that are often not explicitly taught. The 
expectations in school for these general aspects of understanding and learning 
do not match with the experiences of all students, and the transitions are much 
easier for students for whom the school routines and practices are in close 
alignment with those that prevail at home. The difference is generally unfavor- 
able to students of low-income families. 

Recommendation 2: Reading, second language learning , and 
mathematics — Recent reports from the National Research Coun- 
cil, Preventing Reading Difficulties in Young Children , and Im- 
proving Schooling for Language-Minority children: A Research 
Agenda , synthesize strong bodies of research knowledge. A 
similar study on mathematics is currently under way. In each 
case, research is needed now to analyze how the results of our 
knowledge can be implemented in school programs and what 
factors lead to success and difficulties. In reading, research is 
needed on how students become facile at reading complex text as 
they transition to advanced academic subjects such as history, 
social science, mathematics, and science. Research in both short- 
and long-term effects of specific education interventions for 
English language learners is needed, as well as techniques of 
assessment to measure competence, and transition points (a) from 
the first oral language to English, (b) from oral language to 
literacy, and (c) from literacy to the academic discourse of spe- 
cific disciplines. In mathematics, research is needed on why 
students have so much trouble making transitions (e.g., from 
concrete objects to abstract ideas), understanding formal repre- 
sentations, multiplicative reasoning, and essential mathematical 
and statistical concepts such as chance, randomness, and probabil- 
ity. 



High achievement for all students will not be accomplished by policies and 
practices tli'af consider only students’ activities in school without taking account 
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of the competencies that students, especially students from backgrounds of 
poverty, develop in other aspects of their lives. Several recommendations focus 
on ways in which the social arrangements of learning in school, as well as the 
content of school learning activities, need to be studied and understood to 
inform policies and practices that can lead to high achievement for all students. 

Recommendation 3: Organization for learning out-of school — To 
take advantage of learning environments in which children from 
impoverished backgrounds often display more competence than 
in school settings, research is needed to design and test different 
models of after-school and summer programs to motivate, en- 
gage, and benefit low-income children. Work is also needed on 
types and features of after-school opportunities that most effec- 
tively motivate academic achievement and positive self-estima- 
tions; and how to design and test different models of collabora- 
tion between schools and community groups dedicated to pro- 
viding strong learning environments for disadvantaged children. 

Recommendation 4: Organization for learning in-school- — Reten- 
tion, pull-out remediation, tracking, and segregated special 
education programs that stratify by race, class, and gender oppor- 
tunities to learn do not result in high achievement for all stu- 
dents. A more complete inventory of knowledge about effective 
practices for teaching academically challenging curricula with 
groups is needed, both for school populations in general, and for 
heterogeneous groups in particular. Research is needed on ques- 
tions of time for children to master challenging curricula, sup- 
portive school structures, and expectations for the breadth and 
depth of content. Within each of these is the question, do stu- 
dents from middle-income families as well as students from low- 
income, ethnic, and linguistic minority backgrounds benefit 
from each organizational practice? An important area of inquiry 
is whether there are academic benefits to classroom diversity- 
does diversity improve subject-matter learning? (Note: Recom- 
mendation #5 addresses teaching practice aspects of school orga- 
nization for learning.) 

Teaching and Learning 

The second strand of research to promote high student achievement in reading 
and mathematics is teaching as a professional practice. Without improving our 
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understanding of what it will take to produce a well-prepared and professional 
corps of teachers, school improvement will not be possible. Students living in 
poverty and ethnic minorities have been historically underserved by American 
educational institutions and are an increasingly large proportion of the student 
population. No one doubts that teachers will have much to learn in the years to 
come in order to be successful in helping all students reach high levels of 
achievement. There should be a particular concern with producing new knowl- 
edge about connections between professional development and improving 
education for currently underserved populations; namely, children and adoles- 
cents whose experiences and dispositions do not match with the expectations 
and social organization of schools. 

Recommendation 5: Linking changes in teaching practice with 
improved student learning — Information is needed that can guide 
teachers and institutions who want to change their educational 
practice, particularly to reduce inequities in the opportunities of 
students who differ in socioeconomic status, ethnic background, 
and gender to learn successfully. This is especially important 
regarding the achievement of deep intellective competence advo- 
cated in current educational reforms. Such research would exam- 
ine fundamental issues about the nature of teaching and learning, 
including, but not limited to, the importance of the skills and 
knowledge of teachers. Expansion is needed in our knowledge 
and understanding of teaching practices, including teaching tools 
such as assessment, that are successful with students who bring 
different cultural resources to their own and to other students’ 
learning. Research would examine, much more than past research 
has done, issues of what it takes to do effective and successful 
teaching with diverse populations of students. (Note: Recom- 
mendation #4 addresses the school organization context in which 
effective teaching for student learning takes place.) 

Recommendation 6: Linking teachers' professional development and 
teaching practices — Research is needed to understand what effec- 
tive teachers do and how they do it. Successful teaching involves 
not only the exercise of skills and application of knowledge but 
also flexible improvisational adaptation in classroom circum- 
stances. Research is needed to understand the roles of more 
profound knowledge and comprehension of subject-matter 
concepts and methods, both at the level one is teaching and in 
relation to other disciplines and grade levels, as well as the role of 
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understanding processes of students’ learning. Research should 
also investigate how the structure of teachers’ work supports or 
hinders their “on-the-job” learning and what kinds of abilities are 
learned in particular situations that can transfer to other settings 
with different circumstances. 

Recommendation 7: Understanding and supporting successful 
professional development — There is need for a better understand- 
ing of teachers’ development as “professional learning,” and 
understanding teaching as a “learning profession.” The prevalent 
model of learning how to teach— the knowledge goes in during 
teacher education or professional development and then comes 
out to be used in the teachers’ own classrooms— does not account 
for the engagement of teachers themselves in improving the 
practice of their profession. What teachers need to learn to put 
reforms in place is not separable from their actual teaching 
practices or from the development trajectories of their careers. 
Research must examine ways in which people of diverse cultural, 
ethnic, and socioeconomic backgrounds are attracted to careers 
in teaching and how professional development resources can help 
increase and maintain diversity in the teaching profession, while 
continuing to improve practice. Further research on teachers’ 
communities of practice is needed, building on findings that 
norms of responsibility and collegial efforts at professional 
problem solving are the most critical factors in improvement of 
teaching and learning. 



Assessment 

Focusing the research agenda as the Board has suggested means that some 
important topics will not receive much attention. For example, such areas as 
cultural and political contexts of schools, educational policy, and school fi- 
nance; education governance; and learning environments and educational 
technology— all of them areas in which important work might be done if 
sufficient resources become available — would not receive significant attention 
under the Board’s view of priorities. 

Among all the topics that would be deferred under the Board’s identification of 
priorities to achieve high achievement for all students, one that the Board 
would single out as a candidate for inclusion at the earliest opportunity is assess- 
ment of teaching and learning for purposes of accountability. 
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Accountability is an increasingly important issue in educational research and 
practice. Current accountability measures, however, do not match the goals of 
most educational reforms for students of low-income families, especially those 
reforms aimed at improving their complex thinking and participation in activi- 
ties of inquiry and understanding. One issue to be considered is the limitations 
of norm-referenced tests that are conventionally presented in standardized, 
multiple-choice formats. Criterion-referenced measures, aligned with teaching 
and learning standards, may assess the competence of these students more 
productively than norm-referenced tests. Alternative methods that are respon- 
sive and valid guides for instruction of students who come from a background 
of poverty should be developed and studied as soon as adequate funding can be 
attained. 

High Standards of Quality Are Created and 
Upheld 

The single criterion by which any scientific enterprise must be judged is the 
quality of its work. Scientific norms must be known and shared. The expecta- 
tions for explicit hypotheses, sound designs, appropriate measures, sufficient 
data of good quality, and logical analyses must be widely shared. High standards 
must be insisted upon in all areas of a scientific agency’s work-in selection of 
proposals, design of appropriate methodologies, creation of research agendas, 
identification of effective and promising practices, and evaluation of all efforts it 
conducts or supports. 

The primary means by which high standards have been developed and assured 
in federal agencies has been through extensive networks to assure involvement 
of peers. Peer review is much more than a bureaucratic instrument. It is a major 
vehicle of communication between the government and the field, a process 
through which principles about research priorities and technical quality of 
research are clearly articulated, and applied to proposals. In the 1994 legislation, 
Congress made its intent clear that a peer system must be applied to every 
aspect of OERI’s work. The law requires: 

that a system of peer review be utilized by the Office — for reviewing and 
evaluating all applications . . . which exceed $1 million; . . . evaluating 
and assessing the performance of all recipients of grants; . . . cooperative 
agreements and contracts; . . . and for reviewing and designating exem- 
plary and promising programs . . . 




27 



Investing in Learning 




In addition, the law requires OERI to adopt, and the Board to approve: 

such standards as may be necessary to govern the conduct and evaluation 
of all research, development, and dissemination activities carried out by 
the Office to assure that such activities meet the highest standards of 
professional excellence. In developing such standards, the Assistant 
Secretary shall review the procedures utilized by the NIH, NSF, and 
other federal departments or agencies engaged in research and develop- 
ment and shall actively solicit recommendations from research organiza- 
tions and members of the general public. 

While the work of the Board since 1995 has frequently centered on the prepara- 
tion and approval of those standards, the Board has also undertaken a review of 
the set that has been in place longest— standards for approval of grants— to 
determine (a) whether the standards are appropriate and useful; (b) whether 
they contribute to fair and high quality competitions; (c) how the process 
worked and how it might be improved; and (d) what recommendations might 
be made on how to configure and maintain peer review panels. 

In this assessment of operation of the standards in fiscal years 1996 and 1997 and 
in two types of competitions— field-initiated studies and Research Centers— 
Board members learned that as many as a third of reviewers had not conducted 
research in education, even though that is a requirement in the standards. 
Among those who might have had research training and had themselves con- 
ducted research, that training and research experience was in broad topical areas 
related to the competition, but not necessarily in the methods and design of 
research in the proposals. In examining the reviews provided by OERI panel- 
ists, the Board-commissioned study found that most reviews provided little 
depth in their commentaries. Reviews were most detailed about project design 
and significance, least detailed on staffing, budget, and management plans. 
Applicants frequently disagreed with reviewer comments, saw the comments as 
superficial or irrelevant, found a lack of comments about design, and cited a 
lack of examples. Applicants also noted limited explanations and mentioned 
large discrepancies among reviewers. They believed that proposals had not been 
carefully read and said that comments were illegible. 

Standing Panels 

As a result of this review, the Board’s principal recommendation on research 
quality is addressed to the establishment of standing panels. 
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Recommendation 8: Standing panels — Standing panels should be 
established to review proposals for each OERI Institute. These 
would be comprised of 25 to 30 members, but with some over- 
lapping membership, so that problems that cross boundaries can 
receive informed attention and so that members of one panel 
with special knowledge could be invited to serve on another 
Institute panel. 

This is the Board’s most urgent and important recommendation on peer review 
processes, and it can be implemented by OERI on its own authority both easily 
and relatively quickly. Standing panels, frequently used by such organizations 
as NIH, offer the most compelling mechanism the Board could find to improve 
the quality of the review process. Standing panels provide continuity from the 
announcement of government funding opportunities to the decisions on pro- 
posals to fund. They provide an informed group to build areas of research over 
time so that the results are cumulative rather than episodic. Such panels can 
attract experienced members who will agree to serve, because the repeated 
contacts with colleagues are more professionally rewarding than membership in 
ad hoc settings. They can provide a forum where the accumulating knowledge 
can be sifted and interpreted, and new lines of research can be identified. 

Thus, standing panels are a device to attract the very people whose judgment is 
needed to ensure that research proposals are of the highest scientific merit and 
are addressed to high priority national education needs. They can also play a 
crucial role in guiding and evaluating the direction of research. 

The Board has additional recommendations that complement and support the 
standing panels. 

Recommendation 9: Panel membership — Panels should represent a 
broad range of perspectives. They must include members with 
strong disciplinary and methodological expertise. Across OERI’s 
panels, gender, race, ethnicity, and geographic diversity must be 
respected. Panel members should be nationally recognized fig- 
ures. 

Recommendation 10: Standards for panelists— The Board continues 
to support peer review standards adopted by OERI, with Board 
approval, which specify that all reviewers meet three criteria: “(a) 
demonstrated expertise, including training and experience, in the 
\ subject area of the competition; (b) in-depth knowledge of policy 
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or practice in education; and (c) in-depth knowledge of theoreti- 
cal perspectives or methodological approaches in the subject area 
of the competition.” 

Enhancing Quality in Competitions 

The Board has made further recommendations to the Assistant Secretary to 
enhance the effectiveness of reviewers, reduce workload for reviewers and 
applicants, and improve center competitions. For all grant competitions, it is 
important that the quality of research designs be rated by reviewers with appro- 
priate technical expertise. The Board strongly prefers that each proposal be read 
by a minimum of five people. More logistical and other support should be 
provided for reviewers along with more in-depth training, and better formats 
should be created in the technical review form to guide the reviews. Reviewers 
should be expected to provide specific, but not necessarily detailed, feedback to 
applicants. Reducing workload for both reviewers and applicants would en- 
hance the likelihood of accomplishing these changes. For example, the Assistant 
Secretary should consider making grant announcements and appointing submis- 
sion dates earlier in the fiscal year; reducing the number of full applications 
through use of preliminary applications; reducing the number of pages permit- 
ted for center applications and the page limit for attachments; assignment of 
specific primary, secondary, and tertiary reviewers to applications; and conduct 
a small pilot project to determine how technology might be used to support the 
peer review process. 

For center competitions specifically, the Board has urged the Assistant Secre- 
tary to clarify the project design criterion so that reviews address the end 
projects proposed as well as the overall center design; increase weighting for 
management and clearer instructions; and provide planning grants. These 
changes require modifications in regulations. 

The Board also has recommendations for enhancing peer review processes that 
may require changes in the 1994 Act. 

Recommendation 11: Distinguishing between field-initiated and 
directed competitions — OERI should distinguish between field- 
initiated competitions and those that are directed, rather than 
trying to combine the two. 
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Recommendation 12: Funding for peer review — The allowable 
percent of funds to support peer review should be increased so 
that the necessary standing panels may be established and 
logistical support provided. 

Recommendation 13: Definitions — The term “research” should 
have a narrower definition than it has in the 1994 law so that 
the boundaries of focused competitions for research can be 
limited. Research should encompass basic research in educa- 
tion as well as investigations, experiments, and inquiry to 
develop new knowledge or apply tested knowledge. It should 
exclude development, planning, and demonstrations. The 
term “national significance” needs to be clarified through 
regulations or in legislative language so that reviewers under- 
stand that it includes research opportunities, not only impor- 
tant problems identified by educators. 

Work is Collaborative and Rigorous 

Collaborative Research 

As noted in section I, the 1994 Act sets the tone for collaboration in all of 
OERI’s work, including that of the Board in its relationships with the Office 
and the Assistant Secretary. OERI should conduct all its work in ways that 
bring diverse perspectives constructively together. This includes the perspec- 
tives of researchers, educators, policymakers, the public; representatives of 
the nation’s diverse populations and cultures; federal agencies participating in 
the conduct of education research, as well as states, foundations, and the 
private sector. To the extent appropriate for each function, this range of 
perspectives should be represented in all of OERI’s activities, from develop- 
ing agendas, to selection of awardees, oversight, evaluation, and refinement. 

OERI has an important place in research in education throughout the De- 
partment and across the government. This is defined by Congress in broad 
terms in the 1994 legislation. For example, the law sets forth a coordination 
role for the Assistant Secretary and OERI: 

With the advice and assistance of the Board, the Assistant Secretary 
shall work cooperatively with the Secretary and the other Assistant 
Secretaries of the Department of Education to establish and maintain 
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an ongoing program of activities designed to improve the coordination 
of education research, development, and dissemination and activities 
within such Department and within the Federal Government. 

The law goes on to specify the goals of minimizing duplication, maximizing the 
value of federal investment, and enabling entities in education research to 
interact effectively as partners. 

But OERI is limited in its abilities to achieve this role. It provides, for example, 
only a small part of the total support for education research and development. 

A study prepared for the Board estimated U.S. spending for research in educa- 
tion in the range of $900 million to $1 billion through the U.S. Department of 
Education and among foundations. A larger net that includes investments in 
education studies and data collection in other federal agencies such as the De- 
partment of Defense, the Department of Health and Human Services, and the 
National Science Foundation, as well as state and local governments and univer- 
sities might add another billion. With a $2 billion level of annual expenditures 
for education research activities, the nation would be investing less than 0.5 
percent of the total enterprise in educational knowledge-building. However, the 
OERI share represents only about one-fifth of that of the Department and 
about one-twentieth of the $2 billion estimated national education research 
investment. 

Aside from the level of investment is the question of OERI’s span of activities 
in education research. To be blunt, OERI is only one among several agencies 
involved in significant support and or conduct of education research. Within 
the Department, research related to individuals with disabilities and education 
of children with disabilities is considerably larger. The Department of Health 
and Human Services conducts research on learning, family structure, integrated 
service delivery, and funds dissemination activities related to education. The 
Department of Labor funds research on dropouts and illiteracy, and funds 
dissemination activities related to education. The National Endowment for the 
Humanities funds research and dissemination on students’ knowledge of history 
and the humanities. The National Science Foundation has worked on the 
teaching of math and science, NIH in learning disabilities and reading, and the 
Office of Naval Research (ONR) in cognitive and neural science and technol- 
ogy- 

These simple facts have impressed upon the Board that the almost unbounded 
role envisioned for OERI in the 1994 legislation creates unachievable ends. 
OERI is not a monopoly, not the most significant element, not the leading 
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federal influence in several prominent substantive areas. The reality is that 
OERI must carefully balance its own initiatives and its collaboration with other 
federal agencies, foundations, and states and localities. 

In the coordination of Department research and in exercise of its authorities for 
government-wide collaboration, OERI and the Department, nevertheless, have 
vital strengths that all the government agencies can respect, in that they: 

• retain substantial credibility and utility as convenors— they can bring 
people together; 

• maintain unparalleled connections with the nation’s educators and 
education policymakers; and 

• support the policy goal of assuring that the nation’s education resources 
are used effectively to provide opportunities to those individuals in our 
population who have traditionally not been served well. 

OERI’s strengths can often be used particularly well in combination with 
complementary attributes of other agencies. The Board makes the following 
recommendations simply to reinforce the letter and spirit of the 1994 OERI 
law: 



Recommendation 14: Collaboration across federal agencies — The 
Assistant Secretary should extend efforts to join with other 
federal agencies, and perhaps foundations, to collaborate on 
common agendas. 

Recommendation 15: Coordination of research within the Depart- 
ment of Education— The Secretary should encourage, and the 
Assistant Secretary should provide, special attention to perform- 
ing a visible and constructive role in collaboration and coordina- 
tion of education research within the Department of Education. 

The Board does not view these as mere bureaucratic exercises. Every effort 
should be made to create constructive tasks for which it is to the advantage of 
all collaborators to join with the Assistant Secretary. Some examples of what 
the Board members expect might include (1) large areas of research that a single 
agency would not have the resources to undertake alone, (2) syntheses across 
important topics such as development or learning in which the research sup- 
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ported from several perspectives would be tapped, or (3) evaluation of potential 
future activities so that overlapping agency interests can be avoided or exploited 
in purposeful ways. 

One additional aspect of collaboration needs to be strengthened. The Board has 
been troubled to find that the relationship between the OERI-supported re- 
search and development centers and the regional labs is often distant rather than 
collegial and mutually reinforcing. There are concerns that in a period of woe- 
fully inadequate resources, the nation is reaping less than it could gain from its 
investment in the two enterprises. In caricature, the centers are sometimes seen 
as the thinkers and the regional laboratories as interactors with practitioners. 
But this is out of touch with the realities of how practice will improve and how 
comprehensive reform is being implemented effectively. 

Recommendation 16: Linking ongoing research and practice-related 
efforts — The Assistant Secretary should develop constructive 
means through which OERI, the research and development 
centers, and the regional laboratories can function closely to- 
gether to maximize their collective impact. 

Both the centers and the labs need to be thoroughly grounded in practice 
realities, while playing complementary and collaborative roles. The centers may 
emphasize their comparative advantage in broad analysis and conceptual frame- 
works, while the labs may emphasize integration with practitioner settings. But 
that is not enough. Both of these sets of institutions, and OERI, should expect a 
more overarching synthesis of research and the practice of education. OERI 
should create appropriate mechanisms through which that can be achieved. 

This might begin as simply as describing functions and tasks of centers and labs, 
as viewed by the Assistant Secretary, or a description of lab and center work 
mapped onto OERI/Board priorities. 



The Board also wants to associate itself with conclusions from the National 
Directions conference about significant roles for the Board and OERI that 
emphasize the utility of recommendations 14 and 15. Those conclusions were 
that the Board and OERI are particularly suited to: 



• Convene meetings on educational research planning; and on such under- 
lying issues as standards of evidence and methodological progress. There 
are, otherwise, no naturally occurring forums for such discussions that 
transcend specific missions and agendas. 
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• Encourage and coordinate communications strategies, to place the 
accomplishments, promise, and challenges of educational research before 
its professional and public audiences. 

• Monitor the educational research system, and build human and institu- 
tional resources. 

• Build linkages between research endeavors and teachers and other 
educators in the field, through formal consultations, network building, 
professional training programs, systematic translation of research find- 
ings into program designs, and promising implications for the organiza- 
tion and conduct of instruction. 

Synthesis 

Finally, the Board has selected one of these conference conclusions for special 
attention as a recommendation. 

Recommendation 17: Synthesis activities — OERI should support 
synthesis activities across all-important fields of educational 
research, summing up progress continually and drawing implica- 
tions for policy and practice. 

On balance, the government has devoted more of its attention to making 
awards of funds for additional research and development than to making the 
most of what is already known. Synthesis activities can provide a basis for 
identifying implications of research findings for practice, promising research 
that might lead toward improved education practice, or potential new research. 
Some of this work may be especially appropriate for leadership, and even direct 
conduct, by government staff because of the unique perspectives offered across 
the Department and throughout the breadth of the government. 

OERI Staff 

These descriptions of roles for OERI raise an urgent question about the expecta- 
tions held for OERI staff upon whom these responsibilities will fall. To fulfill 
its charge as a government research and research support agency, OERI needs 
staff who can participate as peers in the scholarly community and work with 
users to facilitate the practical application of knowledge. In other federal re- 
search agencies, staff perform significant leadership roles that advance the 
enterprise, and at the same time attract and retain highly trained and capable 
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individuals. Among the variety of staff responsibilities that may be found across 
government research and research support agencies are conduct of research, 
synthesis of research, collaborative efforts with external stakeholders, planning 
and design of cutting-edge research, organization of and participation in peer 
reviews, review of proposals for social utility or agency relevance, and evalua- 
tion of ongoing research. 

Recommendation 18 : OERI staff - — The Assistant Secretary needs 
to determine the types of responsibilities most appropriate to 
accomplish the research functions of OERI, both to advance its 
work, and also to attract, retain, and continuously nurture staff 
with the requisite training, knowledge, and capabilities. 

Several strategies will be necessary to build staff capacity in OERI. First, addi- 
tional staff with demonstrated research accomplishments and potential must be 
attracted. Second, intensive professional development opportunities are needed 
for existing staff, including support for graduate study, details to other scientifi- 
cally oriented research agencies, and newly energized employee development in 
the workplace, as, for example through mentoring, in-house seminars, and 
group projects. Third, staff with generalist skills may need to be reassigned to 
other areas in OERI or in the Department where their experience can be better 
used. 

Design for Rigorous Research 

The National Directions conference participants also concluded that hand-in- 
hand with a sense of focus and collaboration in research must come emphasis 
on more rigorous methods and designs, with particular attention to: (a) rethink- 
ing, reimagining the possibilities of experimental field trials given new technical 
tools, the complexity of the puzzles that must be unraveled, and the persuasive 
power of randomized trials with policymakers and the public; (b) design pro- 
cesses or “engineering” that systematically apply insights of research to the 
development of discrete education programs; and (c) creating a universe of 
reliable syntheses of all important areas of educational research. 

Following these conclusions from the National Directions conference, OERI 
would fit between the research community, the political community, and the 
world of practice. It would not only undertake research and related work for 
which it is especially suited, it would assist all agencies, associations, institu- 
tions, and individuals involved in educational research and improvement to add 
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more value to their own work and to the joint endeavor of learning. The 
goal can be clearly stated: in the future, all education must be based on ideas 
that are subject to validation or invalidation by well-designed, well-executed 
research, and translated into success by well-qualified professionals. 

The National Academy of Education included in its advice to the Board a 
strand on the design of research in education. The Academy’s advice began 
with observations on the usual connotation of “research” and “practice” in 
education research. Historically, the relation between research and practice 
in education has been troubled. The field of education does not have a 
strong, well-established professional community that takes as its charge the 
design and development of practice-relevant theory, products, and proce- 
dures based on established scientific principles and data. There are important 
examples of such design and development work, but it does not occupy a 
sufficiently stable and extensive community of researchers and developers. 
The field does not include a sufficiently established institutional home, form 
of professional identity, or set of incentive structures that can be called on to 
support sustained attention to a continuing, integrated set of activities aimed 
at solving pressing educational problems and developing sound educational 
theory. 

Worst of all, education lacks a well-established tradition of mutual account- 
ability between education research and practice. There are examples of 
collaborations in which researchers and professional educators join in shared 
accountability for educational improvement and advancing research, but 
these are exceptional. In most cases, researchers are expected to study impor- 
tant educational questions, but their work is judged almost entirely by its 
quality as research; the relevance of the work to the details of education 
practice is secondary, despite frequent attempts to document some kind of 
“impact.” On the other side, knowing research, seeking it out, and acting in 
accordance with its results (even when these results challenge some tradi- 
tional and favored ways of doing things) are exceptional rather than norma- 
tive behaviors for working educators. Furthermore, regarding contributions 
to the research literature, professional educators are generally viewed, by 
themselves as well as by researchers, as the objects of study, rather than as 
participants in constructing knowledge and understanding general principles. 

This situation is not new. The history of OERI and its predecessors (the 
National Institute of Education, and the U.S. Office of Education within the 
Department of Health, Education, and Welfare) is in part a tale of trying to 
establish, nurture, and sustain a more productive relationship between 




research— the source of a reliable knowledge base for education improvement— 
and educational practice. The conceptual basis for much of this work, the 
Research-Development-Dissemination-Evaluation (RDDE) model, however, 
embodies a set of assumptions about the way in which research and practice 
should interact that the National Academy of Education, as well as members of 
the Board, question. 

In the past, researchers typically have taken responsibility for producing new 
knowledge that relates to some aspect of learning, pedagogy, or schooling and 
for disseminating that knowledge through the traditional academic venues of 
scholarly journals and meetings. Others, often education practitioners, have 
assumed the responsibility for designing educational products and programs, 
sometimes based on research data but more typically based on experience and 
intuition, in response to specific problems. Pressures to be pragmatic and move 
on to the next problem often mean that practitioners devote little time to 
analyzing how their programs work or how they may be useful in other set- 
tings. 

There is a second dimension of the lack of connection between what we know 
and what we do with what we know. No matter how research is conducted, 
through surveys or ethnographic methods or “design experiments,” we need to 
develop better strategies to ensure that research findings will have a wider 
impact in schools and communities with significant numbers of low-income 
students. For the most part, we have treated the intersection of research and 
practice as one in which researchers transmit the products of research to practi- 
tioners. This situation is ironic, for we know that the transmission model does 
not work for the education of children. So why, then, do we think it should 
work for interactions with teachers? 



The RDDE approach to research and practice has proven to be unproductive 
and myopic. When researchers take little or no responsibility for making things 
work and practitioners eschew development of explanatory systems for how 
and why things work or do not work, neither research nor practice benefits. 
Without a sense of mutual obligation to one another, researchers and practitio- 
ners continue in their own worlds, talking past rather than to one another. 

Problem Solving 



The Academy’s analysis concluded with concrete ideas for a new design for 
“problem-solving” research in education, and the Board’s recommendations are 
built from that advice. 
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Recommendation 19: Collaborative problem-solving research and 
development — OERI should begin development of a new design 
for research that would focus explicitly on solving specific cur- 
rent problems of practice and, at the same time, be accountable 
for developing and testing general principles of education that 
can “travel” to locations beyond those in which the research is 
done. The central idea is to develop a system of support for 
projects in which professional researchers and professional 
educators share in the accountability for achieving success in 
improving educational practices and outcomes. These projects 
may also include program developers, curriculum specialists, or 
policy specialists. Initially, this effort might be launched through 
a “working group” to assist in designing the details about specific 
parts of the priority research agenda for which this model would 
be best suited, the role of OERI and outside contractors or 
grantees, coordination across multiple sites, expectations as to 
scale and length of commitment and the like. 

Board members believe that this proposal will provide powerful new means 
through research and development to grapple with some of the most persistent 
and deeply ingrained problems of our day, and especially the ones contained 
within the Board’s priority for high achievement for all students. All the par- 
ticipants in collaborative problem-solving research and development would 
share in a commitment to and accountability for multiple outcomes of this 
approach. These will include: (1) tangible improvement of participating educa- 
tion systems, responsive to the circumstances of those systems and demon- 
strated by documentable criteria; (2) development of materials, personnel, and 
other resources to support transport of the aims, operational concepts, and 
methods that are developed in the project to other sites; and (3) contributions to 
the research literature that document the results of these efforts and allow for 
alternative explanations of them, so that the results of these projects will ad- 
vance society’s understanding of educational practices and purposes. 

The novelty of this recommendation is in the shared goals and accountability 
for them by all of the participants, rather than separating responsibilities of 
teachers, developers, and researchers. This is not recommended as a replacement 
of support by OERI for other basic research or applied research and develop- 
ment activities. Rather, Board members believe that this kind of integrated 
work should be added to the program of activities supported by OERI and 
should be closely allied with both fundamental and applied research, which the 
Department should continue to support vigorously. 
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Balanced Role 

In summary, the strategic role that the Board sees for OERI cannot overempha- 
size the unique position the Office can occupy in the realm of research aimed at 
improving educational practice and policy. The territory bounded on one end 
by core learning processes identified in cognitive neuroscience, and bounded on 
the other by the engineering of educational improvement, is vast and compli- 
cated. Whereas OERI can enter into collaborative ventures with NICHD and 
NSF in the realm of the basic science of learning, OERI should lead in develop- 
ing knowledge about learning connected to the real world of classrooms, 
schools, communities, and policies. Moreover, it should also take the leading 
role in engineering and evaluating reforms as they begin to affect children. This 
work is ideally situated in OERI, given its strong relationship with the fields of 
practice and policy, for if reforms do not ultimately reach teachers and the act 
of teaching, an impact on children is highly improbable. 

In the Board’s view, OERI would sometimes initiate an agenda and sometimes 
collaborate with agencies such as the NSF in science and mathematics educa- 
tion, NIH in reading, or ONR in learning. This dual role of initiative and 
collaboration would provide ways to extend the depth and purpose of those 
things that OERI undertakes itself, yet tap some of the strong research designs 
that have been followed in other organizations, while building on OERI’s 
excellent connections with the practice community and its ability to establish 
appropriate contexts. The whole would be more than the sum of the parts. 

Mission Is Congruent with Resources 

The ideal education research system would assure congruence between mission 
and resources. It would build capacities for quality and critical mass, recogniz- 
ing that any lesser goal will reduce credibility and support for the efforts. In 
reaching for this objective, the system would recognize the importance of both 
incremental development and firm long-range plans. As its capability grows, 
OERI will substantially increase its technical expertise and outreach, always 
including the skill to capture and integrate fundamental work wherever it is 
conducted and undertake leadership where no superior source exists. 

The current situation does not resemble this ideal. Instead, the Board has found 
longstanding problems from insufficient aggregate resources in OERI, noted 
above. Too few resources have led to spreading available ones thinly over a 
large number of topics and problems. OERI has created and nurtured three 
major types of institutions over a period of three decades— research and devel- 
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opment centers, regional educational laboratories, and ERIC research informa- 
tion centers — each with multiple sites. The resources to make these many 
institutions fulfill the roles for which they were created are extremely con- 
strained. Funds for field-initiated studies and new research initiatives have often 
been negligible or non-existent. 

Recommendation 20: Funding for research in education— Funding 
for education research must be increased dramatically. An in- 
terim target should be the level proposed by the President’s 
Committee of Advisors on Science and Technology of 0.5 per- 
cent of our nation’s expenditures for elementary and secondary 
education— about $1.5 billion annually. This would be a feasible 
target to reach over a 5-year period. 

There is no investment in education that the federal government might make 
that is of equal importance to research. Education research has demonstrated 
over and over again that it can provide useful answers for education practice 
and policy. It is the most vital strategy for lifting student learning to the levels 
we must achieve to maintain our democracy and its vibrant economy into the 
21st century. 



Recommendation 21: Aligning resources and mission — Over the 
next 5 years, OERI should work to make the resources and the 
missions better aligned. Mission must be matched with money. 

If more resources are not forthcoming, the mission-resource mismatch will have 
corrosive and cumulative consequences. The extensive prescription of organiza- 
tional features within OERI may need to be reconsidered if funds are insuffi- 
cient, and in that event, it would be prudent if the Secretary had authority to 
modify the organizational structure. 

Effective use of resources does imply a careful consideration of use of OERI 
staff and internal oversight of research. The Board notes that issues of quality 
across the agency, coordination of work internally and collaboration externally, 
and the substantive development of the research agenda are among issues that 
call for continuing supervision, mentoring, and quality review. It would be 
highly desirable to build into OERI the means for stabilizing a professional 
research function that offers some insulation from constant changes in leader- 
ship and course of direction, even appearances of politically inspired or ideo- 
logical research agendas. 
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Recommendation 22: Research supervision— Some focal point 
should be created by Congress for research leadership that can 
span across administrations. 

In addition to generally insufficient funding, the allocation of resources for 
OERI, particularly to and by the institutes, is heavily controlled by statutory 
and other distribution rules. These rules frustrate responsiveness to new needs 
and circumstances, fractionate limited funds, and inhibit response to new 
national priorities. In a well functioning system, these rules should not be 
necessary. 



Recommendation 23: Allocation requirements — The allocation 
provisions in the 1994 Act for institutes and for types of support 
should be removed. 



V* 




42 



Investing In Learning 



IV. Concluding Observations 

To supplement its recommendations for research in education, the Board has 
some concluding observations about its own work, and particularly about how 
it can best be effective. The legislation that created the Board contains numer- 
ous references to “collaboration.” In addition, the Board is required by law to 
“review regularly, evaluate and publicly comment upon” actions of the adminis- 
tration and Congress. Among provisions of the law is one that says the Board 
must be offered an opportunity to provide written comments on any proposed 
“grants, contract, or cooperative agreements” over $1 million, and those com- 
ments may cover both consistency of the proposed use of funds with the re- 
search priorities and the “methodology and approach of the proposed actions.” 

Taken together, these are potentially powerful authorities, although they must 
be exercised in balance with the resources the Board has available or can attain. 
On one occasion when appropriations exceeded the administration’s budget 
request, the Board was invited to provide an overall analysis and make recom- 
mendations prior to Department decisions about use of these appropriations. 
The Board believes that was an especially effective interchange between the 
administration and the Board and one that permitted the Board to be both 
constructive in its comments and effective in its influence over the subsequent 
actions. However, there have been other occasions, such as preparation of the 
President’s budget and the administration’s proposals for a national voluntary 
test, on which the Board’s involvement came not only after the decisions were 
made but after public debate had begun. These latter examples do not seem to 
demonstrate the collaborative relationship that the law seeks to establish. Even 
worse, perhaps, they fail to take advantage of the counsel the Board was created 
to provide. It is just that structure and those processes that the Board has used 
to prepare this policy statement. This is one concern. 

Of greater importance, however, is the view of Board members that much has 
been accomplished during the past 4 years to serve as a platform for the future. 
The members have learned to work through the diversity of views that Con- 
gress wisely insisted be represented among the appointees. All have come to 
appreciate the potential of sound research in education as a means through 
which all American students can become better prepared for their lives in a new 
millennium. 
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Concluding Observation* 



Members of the National Educational Research Policy and Priorities Board 
appreciate the opportunity that Secretary Riley provided them through ap- 
pointment to the Board to serve American education. They are committed to 
perform, and eager to continue, their special functions in policy and priority 
setting for education research. 
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